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Vehicle detection application can assist in-vehicle surveillance functions and 
have implications for various fields. A vehicle can be identified through the 
license number attached to its license plate, the color and its shape. Vehicle 
detection can make use of multimedia sensors so that the design and detection 
performances can be optimal. Sensor performances are influenced by factors 
such as the number of multimedia sensors, sensor placement, sensor 
positioning, and schemes in case of system failure. This study makes use of 
multimedia sensors with cameras equipped by a phase detection auto focus 
(PDAF) technology which is like a pair of eyes to see an object. This study 
analyses 134 vehicles with number detection and various colors to see the 
effect on the detection and recognition processes. The cars were passed 
through the camera 10 times at a speed of 10-15 km/hour with various camera 
distances and positions. Various values and depths of the images were 
generated. The farther the distance the higher the disparity values. For 


maximum distance of 50 m, disparity is 6.20x106 and image depth is 
16.88x109. Vehicle color influences detection with orange has the best 
accuracy, but the gray has the largest path error value. 
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1. INTRODUCTION 

Target detection and tracking is one of the important applications on the multimedia wireless sensor 
network (MWSN) network which can be used for missing vehicle tracking, determining vehicle location, 
calculating the amount of fuel [1], [2], tracking assets and object [3], war sites monitoring [4], illegal 
immigrants detection at the border [5], gas leak detection [6], forest fire detection [7], and wildlife tracking [8]. 
Target recognition and target search algorithms on MWSN network require sensor nodes that can detect targets 
according to the direction of their movement and manage the information centrally. Target detection and 
tracking algorithms are expected to have energy efficient concern and to be able to track multiple objects. 
Therefore, the MWSN nodes in object detection, specially vehicle detection should be placed and positioned 
at a certain direction at a certain time on the side of the vehicle way and able to work on heavy traffic. This is 
important for tracking targets and predicting the target's presence. So that the target search capability will be 
faster and efficient so that the information management on traffic controller can efficiently manage sensors to 
carry out tasks simultaneously which leads to a more optimal tracking process. 

A vehicle can be identified by its color and the license number for security, odd-even traffic control, 
and electronic payment systems for toll and parking payments [9], [10]. Wavelet decomposition and 
convolutional neural network (CNN) algorithms recently are used for the detection and recognition of vehicle 
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plate numbers and colors and produce an accuracy up to 99.43% for vehicle location determination and 98.9% 
for vehicle plate number recognition [9]. Image enhancement algorithm combined with CNN have been 
introduced in vehicle license number detection in Indonesia by accuracy of 99%. The growing popular you 
only look once (YOLO) algorithm is easily implemented for object detection algorithm with comparable 
97.82% accuracy for plate detection accuracy and 96% for character recognition accuracy [11]. In other case, 
non-standard license plate detection has been implemented in Pakistan. Private number plates should be 
recognized and monitored for several purposes including security as well as a well-developed traffic system 
[12]. Other algorithms such as single shot detection (SSD) algorithm can classify and detect real-time objects 
precisely. This algorithm can train itself by using 1,524 images of motorcycle license plates and then, the 
characters of plate numbers are extracted by using the optical character recognition (OCR) module [13]. While 
the image processing algorithm used aims to detect and track vehicles based on the color and shape of the 
vehicle features with statistical models [14], the target tracking via color information can also be processed by 
using image processing algorithm [15]. 

The system for detecting and recognizing vehicle plate numbers and vehicle colors uses multimedia 
sensor equipped by a camera. These sensors have different capabilities and different prices to achieve the 
desired role and performance [16]. Selection of the type/type of sensor device and placement of multimedia 
sensors have an important role on the highway, which is used as a medium for taking pictures of detecting plate 
numbers and vehicle colors. The detection work process can be done by knowing whether the color of the 
vehicle can affect the detection and tracking of the vehicle plate number. For this reason, it is necessary to 
consider the optimal number of multimedia sensors to be used and the positioning of the multimedia sensors if 
the solution occurs in the event of a system failure [17], [18]. The methodology and achievement of a 
performance of this multimedia sensor installation are for visual information collection, identification, and 
tracking of vehicle plate numbers, and the effect of vehicle color on the recognition and tracking of vehicle 
plate numbers [12]. 

This research makes use MWSN sensor equipped by the camera with phase detection auto focus 
(PDAF) technology, namely autofocus technology that can adjust images based on one object phase. This 
technology is similar to a pair of right and left eyes seeing an object [19]. The sensor network contains 4 sensor 
with 3/32 GB RAM and memory specifications, 5.0 inch IPS screen, 1080x1920 pixels, Android OS 
Marshmallow 6.0.1, Snapdragon 625 chipset (14 nm), 13 MP Dual LED flash camera, and battery: Li-Po 4100 
mAh. Sensors are mounted at a height of 1 m, 2 m, 3 m, and 4 m using poles. 

The object of taking pictures to detect vehicle plates is in the form of 2 cars with white plate numbers 
BK 1801 ZD and black BK 1272 OY. The two cars will pass the camera sensor at a speed of 10 to 15 km/h 10 
times. To collect data on the effect of the vehicle color, 132 cars objects with various colors were used. The 
cars passed camera sensors at a speed of 10 to 15 km/h from 12 p.m. to 1 p.m. when traffic was heavy. The 
impact of this research is the placement of multimedia sensors on the vehicle plate number detection system 
and the effect of vehicle color on the detection process. Optimal sensor placement will have the effect of 
optimal image capture in the form of vehicle plate numbers and vehicle colors so that the data reading time 
will be optimal as well. 


2. THE ALGORITHMS 
2.1. Deep learning 

Deep learning is a sub-section of the scientific field of machine learning [20]—[22]. This field teaches 
a computers be able to perform as humans do. In deep learning, a computer can learn to classify images, sounds, 
text, or videos [23], [24]. In processing, a computer is trained to use a sufficiently large number of labeled data 
sets, and then it can convert pixel values of an image into an internal representation or feature map. Feature 
maps can be used to detect or classify patterns in input. 

The deep learning method approaches data in two sessions, namely training and testing [25]—[27]. In 
a training sessions, we learn feature extraction from each data then distinguish one label from another. In the 
testing session, tested data can be analyzed from the results of a training session. One example of an algorithm 
in deep learning is a CNN [28] and YOLO [29]. 


2.2. Convolutional neural network 

A CNN is one of the deep learning algorithms. CNN is designed to process image data. CNN is aimed 
to classify data labels using by supervised learning method. CNN is limited to processing only grid structures, 
such as digital images and videos. CNN has several parts, such as convolution layer, activation function 
rectified linear unit (ReLU), pooling layer, fully connected layer, and classification, and detection 
output [30]-[32]. 

The convolution layer consists of neurons are arranged by a kernel filter according to the length and 
height of pixels [33]-[35]. Convolution layer is the most important layer in CNN. The convolution is an 
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operation of linear algebra which multiplies the filter matrix. The pooling layer is done after convolution layer 
processing. The pooling process is reducing the number of parameters by down-sampling operation [36], [37]. 
The kind of pooling layer algorithm is max pooling. In max pooling, the matrix value taken is a maximum 
value. The pooling layer can reduce the size of a volume, the number of parameters, and calculations. After the 
max-pooling process, then the layer will be entered into a fully connected layer. This layer connects all neurons. 
At this stage, an object can be classified and detected whether that object is appropriate or not. 


2.3. YOLOVS 

YOLOv5 is the fifth-generation object detection model that was released in April 2020 [38]. In 
general, the architecture of this model is not much different from the previous YOLO generations. YOLOVvS is 
written in the python programming language. YOLOv5 is not C language in previous versions. It makes 
installation and integration into internet og things (IoT) devices more easier. YOLOv5 also uses CNN 
algorithm. In addition, the PyTorch community is also larger than the darknet community. It means the PyTorch 
will get more contribution in the future. 


3. RESEARCH METHOD 

This study uses the research and development (R&D) method in [39] as a program for the detection, 
and recognition of vehicle plates number using original images taken from camera sensors. The flowchart is 
used to make it easier to understand the steps in the research as shown in Figure |. Figure 1 shows the research 
preparation in the form of procurement of vehicles, camera sensors, camera support poles, and other supporting 
staff. At the data collection stage, the activities are in the form of placing camera sensors on the roadside, 
driving a vehicle/car at a speed of 10-15 km/h, and sensors detecting vehicle number plates. The results of 
image detection as input for the software used and the output of the software in the form of image capture 
results in the condition of the vehicle plate being visible and not visible. 


Preparation stage 


Vehicle plate ‘ i Vehicles are set 
detection ensor Placement to run 10-15 km/h 


a | 


Sottware 
testing 


Image capturing 


System 
Analysis 


Result 
Presentation 


Figure 1. Research flowchart 


This study uses materials that support object detection and recognition methods using block diagrams 
which can be seen in Figure 2. Research was conducted in both field experiment (1 and 2) and computer 
processing (3 and 4). Figure 2 can be explained. 

i) The camera position is set at a height of 1 m, 2 m, 3 m, and 4 m. To collect car plate detection data with 
plate numbers "BK1801ZD" and "BK12720Y" by way of the car passing through the camera at a speed of 
10-15 km/h. Both cars passed the sensor 10 times so that 10 frames were taken at each altitude. A total of 
40 frames were captured on the “BK 1801 ZD” and 40 frames for the “BK 1272 OY”. 
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ii) The camera position can be seen in Figure 3. For car color detection data retrieval, carried out during the 
day around 12.00-13.00 WIB where traffic is very heavy and vehicle speed is around 10-15 km/h. The total 
sample taken was 132 cars with various colors, namely gray 11 cars, white 41 cars, silver 19 cars, black 53 
cars, red 4 cars, brown | car, yellow 1 car, blue 1 car, and orange | car. 

iii) The output of these activities is a video recording of the vehicles that are the target of detection and 
recognition. 


Detect car and plate number 

BE 18012D and BE 12720Y 

Detect car and plate number 
132 cars 


www platereco gnizer.com 
YOLO w5 


Computer 


Figure 2. Block diagram of research 


Camera position 


Car position towards sensors 


Figure 3. Camera placement toward the vehicle 


3.1. Hardware 
This study uses MWSN sensor network with 4 sensors with specifications: 
— Lens baseline = 28.3 mm 
— Focal length = 3.7 mm 
— Horizontal field of view = 65.0° 
— Vertical field of view = 50.7° 
— Sensor medium = 4.7x3.5 mm 
— Physical sensor size = 1/3 inch 
— Pitch size = 16.2 mm? 
— Maximum picture resolution = 4160x3120 
— Resolutions = 13.0 MP 


3.2. Software 

Vehicle plate detection using the online application www.platerecognizer.com as shown in Figure 4. 
The detection of vehicle plates is used by the hardware in the form of 2 vehicle objects to detect the black car 
Honda Mobilio plate no. BK12720Y and the white car Toyota Avanza plate no. BK1801ZD. The system will 
detect the license plate number and color of the two cars. Dimensions required for Honda Mobilio dimensions 
are 4,386-4,398 mm longx1,683 mm widex1,603 mm high [40]. The dimensions of the Toyota Avanza are 
4,395 mm longx1,730 mm widex1,665-1700 mm high [41]. Detection of the vehicle color, used a sample of 
132 cars with various colors, namely gray 11 cars, white 41 cars, silver 19 cars, black 53 cars, red 4 cars, brown 
1 car, yellow | car, blue 1 car, and orange | car. 
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The software used for object detection is using YOLOv5. The placement of the shooting position of 

ue detection is by Figure 3. YOLOVS is consist of, 
www.github.com/ultralytics/yolov5 for download YOLOVS sourcecode. 

— Python 3.10.1 
— CUDA 11.3 
— CUDNN 8.4 
—  Pytorch framework 
— Visual studio 2022 
— Anaconda powershell prompt 
All items must be installed in computer and using the anaconda prompt types “python detect.py —source 
data/images/frame_1.png”, then enter as seen in Figure 5. If the file before detection is taken from the data 
subfolder then the detection results can be seen in the runs subfolder as shown in Figures 6 and 7. 


¢ C  @ platerecognizer.com 


PLATE RECOGNIZER Products v Integrations » = Pr 


Accurate License Plate 
Recognition Software 


License Plate Recognition software that works in all environments, 
optimized for your location. 


Get Started 


Figure 4. Appearance of platerecognizer.com 


io 


python —- source 


Figure 5. Appearance of anaconda prompt 
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i scripts = Documents # D canta 
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Figure 6. Input subfolder Figure 7. Output subfolder 
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4. RESULTS AND DISCUSSION 
4.1. Results 

We used the online application www.platerecognizer.com to detect plate numbers BK12720Y and 
BK1801ZD. Detection time for BK12720Y is shown in Figure 8(a) and detection time for BK1801ZD is 
shown in Figure 8(b). The generated accuracy for both figures is shown in Table 1. Table 1(a) shows that 
BK1801ZD was detected in sensor positions of 1, 3, and 4. Table 1(b) shows that BK1272O0Y was detected in 
all positions. 


25 12 — Height 
10 -—- Detection time 


— Height 
—— Detection time 


Time in second 
er me 
Oo uw 


Time in second 


Height in meter 
oN BD 


Height in meter 


uw 


oO 


(a) (b) 


Figure 8. Detection time for (a) BK1801ZD and (b) BK12720Y 


Table 1. Accuration result for (a) BK1801ZD and (b) BK12720Y 
(a) BK1801ZD 
Height (m) 
Plate number detected 1 2. 3 4 Total of frames 
Number of frames | Number of frames | Number of frames | Number of frames 


B1801ZD 5 
BK1801ZD 2 4 - 1 
B1301Z0 1 
BC1801ZD 1 
BK1801Z0 1 - 
B1801Z0 - 1 
BH1901 - 1 
BZ1801ZD - 1 - - 
BH1801ZD - 1 
41891 - 1 
BN1801ZD - - 
B1301ZB - - - 
B1901Z0 - - 1 
01801ZD - - - 1 
BM1801ZD - - - 1 
1 
5 


fe ee re ee enn 


W100120 - - - 
Not detected - - 7 


BK12720Y 3 

EK12720Y 5 1 - 8 12 

BX12720Y 1 

EK1272BY 1 

BL1272BT - 

EK1272BW - - 
E1272BY - - 
ET12720Y - - 
B412720Y - - 
WB 1272 - - 

EK12120Y - - 
6412720Y - - 
L1272BY - - 

EK12728Y - - 


Pee eee eee 
1 


Next, vehicle detections were performed based on the car’s color using 132 cars sample that crossed 
the highway during the experiment. There were various colors such as gray 11 cars, white 41 cars, silver 19 
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cars, black 53 cars, red 4 cars, brown | car, yellow 1 car, blue 1 car, and orange 1| car. Detection result is shown 
in Table 2 and the time detection graph is shown in Figure 9. 

In this study, the YOLOVS5 is implemented for object detection as well. Samples were compiled in 
Figure 10 and Figure 11. The recapitulation of measurements for car detection is presented in Table 3. 


Table 2. Detection results in vehicle plate number based on car color 
No Carscolor Cars amount Plate number detected _ Plate number not detected _ Percentage 


1 Gray 11 7 4 63.64 % 
2 White 41 37 4 90.24 % 
3 Silver 19 16 3 84.21 % 
4 Black 53 48 5 90.57 % 
5 Red 4 4 0 100.00 % 
6 Brown 1 1 0 100.00 % 
7 Yellow 1 1 0 100.00 % 
8 Blue 1 1 0 100.00 % 
9 Orange 1 1 0 100.00 % 
40 
‘A 37.14 
35 oie 
wt. 31.15 Z i 
ee, Ue i Le 
S ral 3 
3 ‘e «#° 26.82 “425.49 : es 
E m. 21.68 23,7 |\ 
£20 ‘, 
§ 17.31 =. 16.78 
615 \, 
7] @ 14.17 = 13.7 
z 11.71 WAL 17.» 40,61 12.61 
10 +-9.03- #95 49.41 
ae, 
5 
0 
Gray White Silver Black Red Brown Yelow Blue Orange 
Color 


ee Min sees Max at Average 


Figure 9. Detection time chart based on cars color 


Figure 11. Compilation results of car detection plate number BK1801ZD 
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Table 3. Recapitulation of measurement for car detection 


No Frame BK 1272 OY BK 1801 ZD 
Accuracy Accuracy 

lm 2m 3m 4m lm 2m 3m 4m 
1 Frame_1 0.92 0.94 0.92 0.93 0.90 0.92 0.88 0.91 
2 Frame_2 0.93 0.94 0.89 0.92 0.90 0.92 0.89 0.91 
3 Frame_3 0.93 0.93 0.90 0.92 0.90 0.92 0.90 0.91 
4 Frame_4 0.93 0.94 0.91 0.92 0.90 0.91 0.87 0.92 
5 Frame_5S 0.93 0.94 0.88 0.93 0.90 0.91 0.88 0.92 
6 Frame_6 0.93 0.94 0.91 0.92 0.91 0.91 0.88 0.93 
7 Frame_7 0.93 0.94 0.92 0.90 0.88 0.92 0.89 0.90 
8 Frame_8 0.92 0.94 0.91 0.89 0.86 0.92 0.89 0.90 
9 Frame_9 0.93 0.94 0.89 0.92 0.89 0.93 0.89 0.91 
10 Frame_l0 0.93 0.93 0.84 0.90 0.88 0.90 0.88 0.92 


0.93 0.94 0.90 0.92 0.89 0.92 0.89 0.91 


Average (93%) (94%) (90%) (92%) (89%) (92%) (89%) (91%) 


Figure 12 shows image compilations from the experiment. The measurement recapitulations for color 
detection are depicted in Table 4. Orange exerts the highest accuracy and blue experienced the worst. 
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Figure 12. YOLOvS detection result for color detection 
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Table 4. Measurement recapitulation for color detection 
No Vehicle color — Accuracy 


1 Gray 0.90 
2 White 0.89 
3 SIlver 0.91 
4 Black 0.91 
5 Red 0.91 
6 Brown 0.87 
7 Yellow 0.89 
8 Blue 0.80 
9 Orange 0.94 


4.2. Analysis 
4.2.1. Convex area 

The area of monitoring must be convex as shown in Figure 13. The PDAF technology can be adjusted 
based on light reading from two different angles until they become one phase. The process is similar to a pair 
of eyes where right side and left side will look to one side. 


A 2 


di 


Focal Length (f) Focal Length (f) 


B 
Camera Baseline (B) 


Figure 13. Camera view of PDAF [42] 


4.2.2. Field of view 

The object must be captured symmetrically into the field of view (FoV) of the camera. In this study, 
there was the difference in height of the camera, namely heights of 1 m, 2 m, 3 m, and 4 m. The illustration of 
the FoV form can be seen in Figure 14. Figure 14(a) shows 1 m camera height, Figures 14(b) to 14(d) show 
the 2, 3 and 4 m height camera where FoV are located higher than the object. 


Figure 14. Camera placement at (a) a height of 1 m, (b) a height of 2 m, (c) a height of 3 m, and 
(d) a height of 4m 


Table 4 shows that highest accuracy at a height of 2 m. Detection results for plate no. BK12720Y is 
0.94 (94%). Detection results for plate no. BK1801ZD is 0.92 (92%). Based on Figure 14, The height of 2 m 
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can cover the whole car dimension. There is a different disparity value according to object movement toward 
the camera. Disparity value, (d) can be calculated using (1) [43]. 


luti 
d — $x resolution (1) 


f[xB 


Disparity shows to the difference of object location seen by the left and right eyes. For the camera, 
disparity, (d) is calculated based on focal length, (f) which is 3.7 mm, camera baseline, which is 28.3 mm and 
resolution which is 13 MP. The calculation result by using (1) is shown in Table 5. 

Furthermore, the depth of image value (Z) can be calculated by using (2) [43]. The f value reflects 
focal point length, B is based line and d is disparity. The calculation result by using (2) is shown in Table 6. 


(2) 


Table 5. Disparity towards distance 


No __ Distance (m) Disparity 
1 5 0.62x10° 
2 10 1.24x10° 
3 15 1,86x10° 
4 20 2.48x 10° 
5 25 3.10x10° 
6 30 3.72x10° 
7 35 4.34x10° 
8 40 4.96x10° 
9 45 5.58x10° 
10 50 6.20x 10° 


Table 6. Disparity toward depth image 


Disparity Depth image 
1 0.62x10° 168.88x10° 
2 1.24x10° 84.44x10° 
3 1,86x10° 56.2910? 
4 2.48x10° 42.22x10° 
5 3.10x10° 33.77x10° 
6 3.72x10° 28.1410? 
7 4.34x10° 24.12x10° 
8 4.96x10° 21.11x10° 
9 5.58x10° 18.76x10° 
10 6.20x 10° 16.88x10° 


4.2.3. Color analysis 

Object detection needs certain features as reference. Color is one of the feature as a reference. There 
are several factors such as noise in an image, natural articulation of objects, blocked by another object, 
complicated object shape, and drastic changes in lighting. Table 3 shows the various types of car colors that 
were sampled in this study with methods that clarify the image, namely: 

— RGB filters (red, green, blue) 

RGB Filters are the basic filter of all images captured by the camera. The RGB filter will separate the 
red, green, and blue components and their combinations so that only the object image appears and the 
background image is filtered. This filter is used for images with red, yellow, blue, and white car samples. 

— CMY filters (cyan, magenta, yellow) 

CMY Filter is a derivative filter of the basic RGB filter. The CMY filter will separate the components 
of the cyan, magenta, and yellow, and their combinations so that only the object image appears and the 
background image is filtered. This filter is used for images with black, brown, and orange car samples. 

—  Colorizations 

Colorizations help black, gray, and white images to become colorful [44], [45]. This is done so that 
the image becomes better and clearer. These colorizations are used for images that have a gray car sample. 
Table 3 shows the gray car sample has an accuracy percentage of 63.64% and this value is the lowest compared 
to the others. This is because each colorization’s performance is calculated based on the peak signal-to-noise 
ratio (PSNR) value. The PSNR formula is: 
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PSNR = 10 logio (=) (3) 


MSE 


Mean square error (MSE) is the similarity parameter between the colored image and the original 
image. The more similar the MSE value is closer to 0. In the object tracking process, the MSE value is affected 
by the path error with (4). 


MSE = [Pease (Ymean)? (4) 


n 


Description: 

Xmean = Object average detection time response x 

Ymean = Object average detection time response y 

n = number of trials (10 times) 

Based on the graph of the response time of detection in Figure 10, the car is gray (x) with Xmean = 21.68 sec and 
the car is black (y) with Ymean = 16.78 sec. By using (4), the path error value for the gray car (x) = 4.84 and the 
path error value for the black car (y) = 3.75. 


5. CONCLUSION 

Vehicle plate detection by MWSN sensor has been performed by using license number and colors. 
Object detection by using YOLOvS produces the best accuray at sensor height of 2 m. Object detection is 
influenced by the position of the sensor and the type of the technology which is represented by the disparity 
values. These values were influenced by the distance of the vehicle to the camera. Based on the calculation 
results, the farther the vehicle is from the camera, the higher the disparity value and the lower the depth image 
value. The disparity value of the maximum distance of 50 m is 6.20x10°. At this condition, the depth image is 
16.88x10°. The detection accuracy of both vehicles is not affected by the detection process time. Fast and long 
detection time yield similar accuracy. However, image transmission within network influences detection 
process. The larger the image size, the longer the transmission time. Meanwhile, object color influences the 
detection, recognition, and tracking performance. Orange has the best accuracy, but the gray has the largest 
path error value. 
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